Field
The present disclosure relates to a display device capable of image distortion compensation and an image distortion compensation method using the same, and more particularly to a display device capable of image distortion compensation through the handling of video signal inputted thereto during the driving of any image display panel, and an image distortion compensation method using the display device.
Description of the Related Art
Recently, with the beginning of enlarging the image of a small display panel to a big virtual image by using an optical device in order to implement virtual reality (VR) and augmented reality (AR), a technology is being used, which compensates image distortion caused by the optical device. In order to compensate the distortion of photos or videos, etc., caused by optics, a technology for reducing the distortion of an original digital image is used, in which an artificially distorted image is calculated in advance through software and is stored in a buffer, etc., and then is transferred to and reproduced by a display device. Here, the artificially distorted image is calculated in advance by a central processing unit (CPU), a graphics processing unit (GPU), general-purpose computing on graphics processing units (GPGPU), etc. Also, in order to compensate the distortion of photos or videos, etc., caused by optics, a technology is used which stores the image in the form of a file by converting the image into a distorted image in consideration of a specific optical device, and then transfers the converted image to the display panel and reproduces it. Such an image conversion process requires complex operations and frequent video memory access, so that the image conversion is difficult to perform in real time. Therefore, a method is mainly used which generates in advance video information to be transmitted to the display device by means of a separate software content manufacture tool. However, the video signal to be transmitted to the display device has been already distorted. Therefore, when the video signal is displayed on a screen of a general display device, there may occur a problem that the image is recognized as an abnormal image. Meanwhile, in order to compensate the image in real time, as with Oculus Rift, a central processing unit of a desktop level and parallel processing of a graphics processing unit should be used. However, in order to compensate the image in the display device in real time, a reverse process almost the same as the process of distorting the video signal should be performed. Therefore, when a software method is used, a powerful CPU, a powerful GPU, or a powerful GPGPU is required, so that it is inevitable that cost and power consumption are increased.
Unlike a panel used in a typical monitor, a display panel used in a head mounted display (HMD), a head up display (HUD), or a Pico-Projector, etc., magnifies and projects, as shown in FIG. 2, a small image by using an optical device. Therefore, when a user sees a final image, various distortions may occur. Particularly, a pincushion or barrel distortion occurring when a high magnification optical device is used to enlarge a small image to a big virtual image causes the edge of an inputted image to be warped, affecting the image quality. Accordingly, such a distortion is an obstacle when a user enjoys the video information.
Also, a perspective distortion or aberration distortion, etc., may occur according to the design of the optical device in a complicated manner.
The perspective distortion causes a distorted image (Keyston) like a trapezoid instead of a rectangle due to the disproportional horizontal and vertical ratio of the image. A high magnification lens may result in chromatic aberration in which a color to be displayed is distorted due to a refractive index difference according to the wavelength of light. This distortion just has a degree of difference and is inevitable when using the optics, so that the distortion cannot be completely removed. Therefore, the distortion is intended not to be recognized by human. For this purpose, as shown in FIG. 3, a method, etc., is generally used which changes the design of the optical device by using an expensive lens made through a combination of a convex lens and a concave lens, a prism, a mirror, etc., or compensates and improves the image through digital image processing.
The technology for compensating the image distortion caused by optics is mainly used to change an image stored in a state of already being distorted by photographing by a camera into a digital image and to compensate the digital image in the future through a software algorithm by means of a digital signal processing device, etc., such as a graphics processor or a digital signal processor (DSP), or is mainly used to improve the image quality of the display which displays an image on a curved screen by using a cathode ray, etc. Recently, the technology for compensating the image distortion caused by optics has been used to compensate the distortion of results photographed by a digital camera through the image processing and to store in a storage. As a flat panel display displaying directly pixels like an LCD, OLED, etc., is generally mainly used, there was no requirement for the optical device with the exception of the application of a projector, etc. However, recently, with the beginning of enlarging a small display panel to a big virtual image by using an optical device in order to implement virtual reality and augmented reality, a limited image distortion compensation technology is used.
The distortion caused by optics shows in a complicated manner the distortion caused by the lens, the perspective distortion caused by a difference between the panel where pixels are displayed and a space where the image is displayed, etc. Therefore, the distortion caused by optics shows non-linear characteristics. When the distortion caused by optics is compensated by using the digital image processing, a large amount of operations is required. Also, with the increase of the image resolution to be processed, it is difficult to perform the compensation in real time due to a memory bandwidth limit, a response speed limit, the increase of the amount of operations, etc.
The compensation of the pincushion/barrel distortion by the distortion caused by the lens is frequently based on tangential modeling. In this case, a Cartesian coordinate system is converted into a spherical coordinate system, and a non-linear ratio of a distance and an angle from the center of the lens is calculated through lens modeling by approximating the ratio by means of high-order polynomial operations. As the distortion becomes more complex like a mustache shape, the higher-order polynomial operation is required. The coefficient of the polynomial is used by calculating a distortion constant value extracted from a reference image such as a grid pattern or a calibration target point, etc. A way to reduce the amount of operations by forming pre-computed results in the form of a look-up table in order to easily perform the coordinate conversion, modeling calculations, interpolation, etc., a way to establish a hardware logic performing complex operations such as a trigonometric function or a square root by using COordinate Rotation DIgital Computer (CORDIC), and a way to operate In the form of a linear approximation in accordance with each area by applying a grid to the image have been used individually or in a complicated manner.
However, since the display device could not perform the compensation in real time, the conventional compensation technology is used to handle the video source in advance before transmitting the video information to the display device. A method is used which transmits the already handled video to the display device, and then compensates the distorted image in the future by using optical methods.
Among a variety of aberrations distortions, in particular, the chromatic aberration distortion is compensated by a method which, similarly to the compensation of the pincushion/barrel distortion, performs the high-order polynomial operations through the lens modeling by means of a distortion variable based on R/G/B colors, or blurs the boundary of the image by applying a boundary detection algorithm through use of a phenomenon in which a high frequency region of the image, i.e., a boundary of the image is noticeably recognized, and then increases the sharpness or changes the RGB value of the pixel. Likewise, a method for compensating the distortion through a separate conversion process has been mainly used.
The keyston caused by the perspective distortion is compensated by a method which obtains a projection matrix by using a corner point of a projected image and an operation to which a conversion matrix has been applied using a vanishing point, and performs a backward mapping, or by a method which removes or interpolates pixels in a certain ratio by calculating a ratio in which the image is tilted. Unlike the lens distortion, the keyston caused by the perspective distortion is compensated by a method for controlling a projection direction in a mechanical manner.
To compensate the complex distortions, a method has been used which performs the compensation in order in a pipeline manner by applying algorithms for compensating the distortions respectively. Also, each of the compensation algorithms uses a parallel processing capability of the graphics processor or applies a special hardware design technology to which the pipeline is applied. Each of the distortion compensation algorithms may be designed in such a manner as to apply a technology for approximating the complex operations by using an infinite series or Newton's method and to minimize a division and a multiplication, which have a large computational burden. Also, each of the distortion compensation algorithms minimizes the amount of operations or memory access by using temporal/spatial locality of the pixel. Such a method is not performed in the display device and the video information to be displayed on the screen is handled and then is transmitted to the display device.
In order that a small mobile device uses a technology for mitigating the various distortions caused by optics or uses a technology in which the video signal which has been already distorted and transmitted to the display device is converted again in conformity with the environment of the user who looks at the display device, the compensation technology with a low power and a low cost is required. However, there is a difficulty to solve by the conventional method.